Lecture 5: Model selection and assessment

Y = β0 + (β1 + β2)X1 + and we may get a good estimate of Y estimating 2 parameters instead of 3. Our estimate will be a bit biased but we may lower our variance considerably creating an estimate with smaller expected prediciton error than the least squares estimate. We won’t be able to interpret the estimated parameter, but our prediction may be good. In subset selection regression we select a number of covariates to include in the model. Then we look at all possible combinations of covariates and pick the one with the smallest RSS. Consider the prostate cancer data set presented in the HTF book, available in the ElemStatLearn R package (Figure 1). Notice that residual sum of squares